Learning dialogue policies using state aggregation in reinforcement learning
نویسندگان
چکیده
The learning of dialogue strategies in spoken dialogue systems using reinforcement learning is a promising approach to acquire robust dialogue strategies. However, the trade-off between available dialogue data and information in the dialogue state either forces information to be excluded from the state representations or requires large amount of training data. In this paper, we propose to use dynamic state aggregation to efficiently learn dialogue policies using less data. State aggregation reduces the size of the problem to be solved. Experimental results show that the proposed method converges faster and that in case of data sparseness, the proposed method is less sensitive to atypical training examples.
منابع مشابه
Hybrid Reinforcement/Supervised Learning for Dialogue Policies from COMMUNICATOR data
We propose a method for learning dialogue management policies from a fixed dataset. The method is designed for use with “Information State Update” (ISU)-based dialogue systems, which represent the state of a dialogue as a large set of features, resulting in a very large state space and a very large policy space. To address the problem that any fixed dataset will only provide information about s...
متن کاملAutomatic learning of dialogue strategy using dialogue simulation and reinforcement learning
This paper describes a method for automatic design of human-computer dialogue strategies by means of reinforcement learning, using a dialogue simulation tool to model the user behaviour and system recognition performance. To the authors’ knowledge this is the first application of a detailed simulation tool to this problem. The simulation tool is trained on a corpus of real user data. Compared t...
متن کاملAn ISU Dialogue System Exhibiting Reinforcement Learning of Dialogue Policies: Generic Slot-Filling in the TALK In-car System
We demonstrate a multimodal dialogue system using reinforcement learning for in-car scenarios, developed at Edinburgh University and Cambridge University for the TALK project. This prototype is the first “Information State Update” (ISU) dialogue system to exhibit reinforcement learning of dialogue strategies, and also has a fragmentary clarification feature. This paper describes the main compon...
متن کاملUsing Reinforcement Learning to Build a Better Model of Dialogue State
Given the growing complexity of tasks that spoken dialogue systems are trying to handle, Reinforcement Learning (RL) has been increasingly used as a way of automatically learning the best policy for a system to make. While most work has focused on generating better policies for a dialogue manager, very little work has been done in using RL to construct a better dialogue state. This paper presen...
متن کاملEvaluating State Representations for Reinforcement Learning of Turn-Taking Policies in Tutorial Dialogue
Learning and improving natural turn-taking behaviors for dialogue systems is a topic of growing importance. In task-oriented dialogue where the user can engage in task actions in parallel with dialogue, unrestricted turn taking may be particularly important for dialogue success. This paper presents a novel Markov Decision Process (MDP) representation of dialogue with unrestricted turn taking an...
متن کامل